# Low Perplexity

DUO is a pre-trained model for text generation, which can be used for masked language modeling tasks. It is trained on the OpenWebText corpus and has good performance.

The RWKV-7 model in flash-linear attention format, supporting English text generation tasks.

OpenCALM is a 3B-parameter version of the decoder-only language model series developed by CyberAgent, pretrained on Japanese datasets.

SanBERTa is a RoBERTa model trained on Sanskrit text, specifically designed for Sanskrit text processing tasks.

Large-scale Russian pre-trained Transformer language model based on GPT-2 architecture, trained by SberDevices team

A Malay masked language model based on the BERT architecture, trained on the Malay subset of the OSCAR dataset, supporting PyTorch and TensorFlow frameworks.

GPT-fr is a French GPT model developed by Quantmetry and Laboratoire de Linguistique Formelle (LLF), trained on a large and diverse corpus of French texts.